Fixes for file format security issues #4283
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jhendersonHDF
added
Merge - To 1.14
jhendersonHDF
requested review from
lrknox,
derobins,
byrnHDF,
fortnern,
qkoziol,
vchoi-hdfgroup,
bmribler,
glennsong09,
mattjala and
brtnfld
as code owners
jhendersonHDF
force-pushed
the
aws_security_fixes
branch
from
38607e5
to
a070c88
Compare
jhendersonHDF
commented
Mar 29, 2024
| *------------------------------------------------------------------------- | ||
| */ | ||
| bool | ||
| H5T_noop_conv(const H5T_t *src, const H5T_t *dst) |
There was a problem hiding this comment.
The changes in this file mostly come from merging my cleanup with the same cleanup from @qkoziol. This appears to be the main change that was in this file.
jhendersonHDF
commented
Mar 29, 2024
| @@ -589,12 +589,12 @@ dump_table(hid_t fid, char *tablename, table_t *table) | |||
|
|
|||
| PRINTSTREAM(rawoutstream, "%s: # of entries = %d\n", tablename, table->nobjs); | |||
| for (u = 0; u < table->nobjs; u++) { | |||
| H5VLconnector_token_to_str(fid, table->objs[u].obj_token, &obj_tok_str); | |||
There was a problem hiding this comment.
This function is inside a debugging #ifdef, so looks like it was just never updated.
jhendersonHDF
commented
Mar 29, 2024
| @@ -388,6 +389,13 @@ H5O__dtype_decode_helper(unsigned *ioflags /*in,out*/, const uint8_t **pp, H5T_t | |||
| if (H5_IS_KNOWN_BUFFER_OVERFLOW(skip, *pp, actual_name_length, p_end)) | |||
| HGOTO_ERROR(H5E_OHDR, H5E_OVERFLOW, FAIL, "ran off end of input buffer while decoding"); | |||
|
|
|||
| /* Check for duplicated field name */ | |||
| for (memb_idx = 0; memb_idx < dt->shared->u.compnd.nmembs; memb_idx++) | |||
There was a problem hiding this comment.
I wonder if we can do this check at the end of decoding a compound datatype and in some efficient way rather than doing it for every single member. That seems like a real performance killer for large compounds.
There was a problem hiding this comment.
I believe that it'll be O(n^2) either way... :-/
There was a problem hiding this comment.
What if we build a temporary hashmap indexed by hashing the member name and use it for lookups?
There was a problem hiding this comment.
Not something that really needs to hold this PR up, but there are probably some ways we can improve on this.
jhendersonHDF
commented
Mar 29, 2024
| *------------------------------------------------------------------------- | ||
| */ | ||
| herr_t | ||
| H5G__ent_to_link(H5O_link_t *lnk, const H5HL_t *heap, const H5G_entry_t *ent, const char *name) |
There was a problem hiding this comment.
This function appears to have been moved to H5Gent.c, while H5G__ent_convert() from H5Gent.c was renamed to H5G__link_to_ent() and moved into this file.
There was a problem hiding this comment.
Yes, made more senes with what the routines were doing
derobins
approved these changes
Mar 31, 2024
qkoziol
approved these changes
Apr 1, 2024
lrknox
pushed a commit
to lrknox/hdf5
that referenced
this pull request
Apr 1, 2024
derobins
added a commit
that referenced
this pull request
Apr 1, 2024
* Relaxed behavior of H5Pset_page_buffer_size() when opening files (#4280) This API call sets the size of a file's page buffer cache. This call was extremely strict about matching its parameters to the file strategy and page size used to create the file, requiring a separate open of the file to obtain these parameters. These requirements have been relaxed when using the fapl to open a previously-created file: * When opening a file that does not use the H5F_FSPACE_STRATEGY_PAGE strategy, the setting is ignored and the file will be opened, but without a page buffer cache. This was previously an error. * When opening a file that has a page size larger than the desired page buffer cache size, the page buffer cache size will be increased to the file's page size. This was previously an error. The behavior when creating a file using H5Pset_page_buffer_size() is unchanged. Fixes GitHub issue #3382 * chore: improve error message (#4287) close #192 * changed to if string contains instead (#4286) * Fix off_t straggers (#4291) Convert off_t to HDoff_t (mainly for Windows): * h5jam * h5unjam * chunk_info test * Add missing foreach for VFD and Pasthrough runs (#4292) * Fixes for file format security issues (#4283) * Add configure options for disabling extension features (#4277) Add configure option to enable or disable extension features in general Add configure option to enable or disable _Float16 support Add new config options to various settings files * Add documentation (H5FD) (#4269) * Use AOCC 4.2 and OpenMPI 4.1.6 (#4290) * Fix problems with background buffers and array datatypes (#4218) * Fix bug in array conversion with strided background buffer. Convert some memmove calls to non-overlapping buffers to memcpy. * Revert inappropriate use of mempy to memmove in H5T__conv_array * Add testing * Add RELEASE.txt note and overwrite test case. * Synch workflows and require apt-get update (#4294) * Fix Figure 9. table format in HDF5 Groups User Guide (#4295) --------- Co-authored-by: Dana Robinson <[email protected]> Co-authored-by: H. Joe Lee <[email protected]> Co-authored-by: Scot Breitenfeld <[email protected]> Co-authored-by: Allen Byrne <[email protected]> Co-authored-by: jhendersonHDF <[email protected]> Co-authored-by: bmribler <[email protected]> Co-authored-by: Neil Fortner <[email protected]>
qkoziol
pushed a commit
to qkoziol/hdf5
that referenced
this pull request
Apr 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses file format security issues detected via fuzzing.
Credit: Amazon Web Services and @qkoziol